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Abstract  (contin 


ptimizing  the  decision  policies  at  the  nodes  of  a decision  tree.  Also  describe^ 
are  contributions  to  the  modeling  of  communication  channels  with  memory  and  the 
investigation  of  models  of  database  query  languages. 


Among  the  papers  published  on  the  research  performed  under  this  grant,  the 
following  two  papers  present  substantive  overviews  of  the  research  and  place  the 
work  in  the  perpective  of  other  contributions  in  the  areas  discussed.  Both 
papers  include  extensive  lists  of  references.  The  two  papers  are:  Laveen  N. 
Kanal,*tProblem-Solving  Models  and  Search  Strategies  for  Pattern  Recognition1^ 

IEEE  Trans,  on  Pattern  Analysis  and  Machine  Intelligence,  vol , 1,  No.  2,  p,193-2dl, 
April  1979;  and  Laveen  N.  Kanal,  A.R.K.  Sastry,  ▼Modeling  Channels  with  Memor>| 
and  their  Applications  to  Error  Control1*,  Proc . of  the  IEEE,  vol.  66,  No,  7, 
p.  724-744,  July  1978.^.% 
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for  the  design  of  hierarchical  statistical  classifiers  includes  use  of  state- 
space  ordered  search  algorithms  and  introduction  of  heruistics  to  cut  down  the 
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1 Sept.  1977  to  28  Feb.  1979  can  be  categorized  under  the 
following  topics: 

1.  Contributions  to  Pattern  Analysis  and  Classification. 

1.1  New  approaches  to  Search  and  Extraction  of  Structure  in 
Patterns . 

1.2  New  Methodology  for  Statistical  Classification. 

2.  Contributions  to  Modeling  Communication  Channels  with 

Memory  : Extensive  Survey  summarizing  research  performed 

by  ourselves  and  others  on  Modeling  Communication  Channels 
with  memory. 

3.  Modeling  and  Analysis  of  Data  Base  Systems. 

3.1  Investigation  of  Models  of  Data  Base  Query  Languages. 


Summary  of  Progress 

Some  desired  objectives  for  a structural  pattern  analysis 
procedure  are:  1)  pattern  analysis  should  be  able  to  proceed 
in  a bidirectional  data-directed  and  model -di rected  manner  with 
primitive  extraction  and  structural  analysis  coupled;  2)  the 
analysis  should  not  be  restricted  to  a cannonical  left-right  scan 
of  the  entire  data  but  should  be  non-left-right,  selective,  and 


focus  on  prominent  morphs;  3)  multiple  and  ambiguous  interpre- 
tations should  be  developed  on  a best-first  basis,  with 
ambiguity  permitted  in  both  segmentation  and  structural  analysis, 

A structural  analysis  paradigm  which  realizes  the  above 
enumerated  objecti ves-not  satisfactorily  addressed  previously- 
was  developed  with  partial  support  of  this  grant  and  reported 
in  the  PhD  dissertation  of  Stockman  [1977].  This  nondi rectional 
approach  begins  by  identifying  certain  prominent  primitive 
components,  which  can  be  reliably  extracted  and  then  uses  model- 
directed,  data-conf i rmed  search  for  the  remaining  pattern 
structure . 

Because  of  the  correspondence  between  Context-Free 
Grammars  and  AND/OR  graphs,  the  new  nondirectional  analysis 
algorithm  can  bw  viewed  either  as  a "problem  solver"  or  a parser, 
When  applied  to  game  trees,  the  algorithm  is  competitive  with 
although  different  from  the  8 tree  pruning  procedure  in 
efficiently  producing  the  minimax  solution  tree. 

A topic  left  incomplete  in  the  above  work  (done  under 
this  grant  in  the  preceeding  period)  was  the  analysis  of  the 
performance  of  the  non-di recti onal  analysis  algorithm  and  its 
comparison  with  ou8  search  for  evaluation  of  mini-max  trees. 
During  the  current  grant  period,  further  work  was  done  on  this 
topic  leading  to  the  submission  of  a paper  entitled  "A  Minimax 
Algorithm  Better  than  o-6  ?"  by  G.  Stockman  which  has  been 
accepted  for  publication  in  Artifical  Intelligence.  The 
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paper  shows  that  the  new  algorithm  SSS*  is  more  efficient  than 
a-8  in  the  sense  that  SSS*  never  evaluates  a node  that  a-B 
can  ignore.  Moreover,  for  practical  distributions  of  tip  node 
values,  SSS*  can  expect  to  do  strictly  better  than  a0  in  terms 
of  average  number  of  nodes  explored.  In  order  to  be  more  informed 
than  a-£  , SSS*  sinks  paths  in  parallel  across  the  full 

breadth  of  the  tree.  The  penalty  for  maintaining  these  alternate 
search  paths  is  an  increase  in  storage  requirements  relative  to 
aB  . (Some  execution  time  data  indicates  that  in  some  cases 
the  tradeoff  of  storage  for  execution  time  may  be  favorable  to 
SSS*.)  Once  again,  a classical  tradeoff  between  storage  space 
and  execution  time  has  been  achieved. 

Additional  analyses  of  the  "average"  performance  of 
SSS*  have  been  done  during  this  period.  It  seems  likely  that 
a hybrid  SSS*  - “8  procedure  would  be  worthwhile, 
could  be  used  to  a certain  depth  to  avoid  SSS*  storage  problems, 
and  then  SSS*  could  be  used  effectively  to  save  on  tip  evaluations. 
Alternatively  SSS*  could  be  used  at  the  shallow  level  and 
at  deeper  levels  of  the  tree. 

Under  the  category  of  new  methodology  for  Statistical 
Classification  and  Modeling,  research  continued  on  the  Synthesis 
of  Multistage  Multiclass  Classification  Structures.  In  our 
earlier  work,  ordered  search  algorithms  for  S-admissible  and 
B-admi ssibl  e strategies  for  multiclass  pattern  classification 
were  introduced  and  developed.  Under  the  present  grant,  as 
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detailed  in  Kulkarni  and  Kana1(1978),  it  was  shown  that  for 
certain  parametric  probability  functions  it  is  possible  to  derive 
tight  bounds  on  the  class  accessible  from  a given  node  in  the 
state  space  graph  representation  of  the  classification  tree. 

The  paper  also  shows,  by  example,  how  admissible  search  of  a 
state-space  graph  for  classification  improves  on  the  usual 
decision  tree  approach  (in  which  after  reaching  a terminal, 
no  logical  strategy  exists  for  reconsidering  classes  possible 
discarded  earlier). 

In  an  M.S.  thesis  [A.  Dandapani,  1979],  we  have  further 
investigated  the  Bayes-Admissible  version  of  the  B-admissible 
Search  Strategy  and  also  experimented  with  the  heuristics  for 
clustering  decision  rules  and  defining  efficient  decision  rules 
developed  in  our  earlier  research  on  hierarchical  classifiers. 

A phased  approach  to  tree  design  involves  procedures  for 
the  specification  of  (a)  a tree  skeleton  or  hierarchical  ordering 
or  the  class  labels,  (b)  a feature  assignment,  i.e.,  the  choice 
of  feature(s)  to  be  measured  at  each  non-terminal  node,  and 
(c)  the  decision  rule  to  be  used  at  each  non-terminal  node. 


Even  for  reasonable  size  multistage  multiclass  problems,  an 
optimal  tree  design  by  exhaustive  search  of  all  the  above  three 
aspects  is  not  feasible. 

Assuming  that  we  have  obtained  the  tree  structure  and 


the  features  to  be  used  at  each  node,  an  optimal  decision 
policy  at  each  node  can  be  obtained  from  a dynamic  programming 
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(DP)  procedure.  In  our  earlier  work  under  this  grant,  we  intro-  • 
used  the  heuristic  concept  of  an  "efficient"  decision  rule  to 
enable  us  to  reject  a large  number  of  rules  prior  to  their 
use  in  a DP  procedure.  Also,  groups  of  decision  rules  known 
to  be  dominated  by  a given  rule  could  be  rejected  through  a 
branch  and  bound  technique  without  having  to  evaluate  the  per- 
formance of  each  rule  individually. 

The  thesis  [Dandapani,  1979]  and  the  paper  [Dandapani  & 
Kanal,  1979]  report  experimental  results  using  leukocyte  data, 
showing  respectively,  (1)  the  improvement  in  classification 
performance  on  indepent  test  samples,  when  a decision  tree  is 
searched  as  a state  space  suing  the  B-admissible  strategy  and 
(2)  the  reduction  achieved  in  decision  rules  by  using  the 
"efficient"  decision  strategy  in  desgining  a decision  tree 
classifier. 

In  the  area  of  modeling  communication  channels  with 
memeory,  we  basically  wrapped  up  the  research  performed  by  us 
on  this  subject  under  AFOSR  sponsorship  by  writing  an  extensive 
survey  and  commentary  [Kanal  & Sastry,  1978]  covering  the 
significant  work  in  the  field.  The  paper  examines  the  work  done 
on  channel  models  in  the  perspective  of  the  applicability  to  error 
control . 

Work  on  the  development  of  a theory  for  doing  logical 


inference  on  data  bases  was  initiated  during  this  grant  period 
[Jacobs  and  Lavine(1978) , TR699].  A formal  theory  for  relational 


databases  was  developed  using  the  theory  of  models  from  mathematical 
logic.  Criteria  were  established  for  determining  the  querying 
power  of  query  languages.  We  also  worked  on  the  early  stages  of 
a project  to  develop  a comprehensive  logical  model  for  databases 

which  included  as  a special  case  relational,  hierarchical, 

j 

and  network  databases.  The  three  standard  types  of  queries, 
logical  queries,  query  by  example,  and  query  by  algebra,  were 
extended  to  cover  all  three  types  of  data  bases  and  the  equivalence 
of  the  three  types  of  query  languages  was  proved.  This  area 
of  our  research  has  as  its  goal  the  development  of  inference 
techniques  for  probabilistic  hypothesis  evidence  database 
networks . 
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PAPERS,  REPORTS,  AND  PRESENTATIONS  related  to  Research 
for  period  1 Sept.  1977  - 28  Feb.  1979 


Papers 

Kanal,  Laveen  N.,  (1977)  "Hierarchical  Classifier  Theory  and 
Interactive  Design",  p.  301-322,  in  P.  Krishnaiah  (ed.). 
Application  of  Statistics,  North  Holland  Publishing  Co., 

Dec.  1977: 

Kanal, Laveen  N.  (1977),  "Some  Current  Concepts  and  Problems  in 
Pattern  Classification  and  Feature  Extraction",  Bull.  Int.  Stat. 
Inst.  Dec.  1 977,  X LV  1 1 ( 2 ) , p.  464-488. 

Kanal,  Laveen  N.,  (1979)  "Problem  Solving  Models  & Search 
Strategies  for  Pattern  Recognition",  IEEE  Trans,  on  Pattern 
Analysis  & Machine  Intelligence,  Vol  . 1 , no^  T]  pp~!  193-201  , 

April  1979. 

Kanal,  Laveen  N.  and  A.R.K.  Sas try ( 1 978 ) , "Models  for  Channels 
with  Memory  and  their  Applications  to  Error  Control",  Proceedi ngs 
of  the  IEEE, Vol.  66,  No.  7,  pg.  724-744,  July  1978. 

Kulkarni,  Ashok  V.,  Kanal,  Laveen, M.  ( 1 978)  "Admissible  Search'' 
Strategies  for  Paramteric  and  Nonparametri c Hierarchical 
Classifiers",  Proc.  of  Fourth  Intern.  Joint  Conf.  on  Pattern 
Recogni tion,  Nov.  1978. 

Lemmer,  John,  F.  (1977),  "A  return  to  Probabilities  in  Computer 
Assisted  Medical  Diagnosis",  in  Proc  I nternl . Conf . on  Cybernetics 
and  Society,  Sept.  1977,  pp.  612-616,  IEEE  Catalog  No.  L77CH1259- 
I SMC . 

Stockman,  George,  (1977),  " Non-Di recti onal  Structural  Pattern 
Recognition  Using  a Problem  Reduction  Representation"  in  Proc . 
Internl.  Conf.  on  Cybernetics  and  Society,  Sept.  1977,  p.  601-604. 
IEEE  cat.  # L77CH1 259-SCMC . 

Stockman,  George ( 1 978) , "Defining  and  Extracting  Waveform  Prim- 
itives for  Linguistic  Analysis"  in  Proc.  of  Fourth  Internl.  Joint 
Conf,  on  Pattern  Recognition,  p601-604,  Nov.  1978. 

Stockman,  George(1979) , "A  Minimax  Algorithm  better  than  Alpha- 
Beta",  accepted  for  publication  in  a 1979  issue  of  Artificial 
I ntel 1 i gence . 
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PAPERS,  REPORTS,  AND  PRESENTATIONS 


Reports 

Dandapani,  Ajit  (1979),  Hierarchical  Classifier  Approach  to 
Leukocyte  Classification,  Master  of  Science  these.  Submitted 
February  1979. 

Dandapani,  Ajit  and  Laveen  Kanal  (1979),  "Selecting  'Efficient' 
Decision  Rules  in  Designing  Decision  Tree  Classifiers:  Some 
Experimental  Results",  Report,  Lab  for  Pattern  Analysis,  Dept, 
of  Computer  Science,  U.  of  MD,  submitted  for  publication 
April  1979. 

Jacobs,  Barry,  and  David  Lavine  (1978),  On  Completeness  of 
Database  Query  Languages,  Computer  Science  Tech.  Rept.  699, 
Univ.  of  Maryland,  College  Park,  Nov.  1978. 

Kanal,  Laveen  N.,  Some  Current  Concepts  and  Problems  in  Pattern 
Classification  and  Feature  Extraction,  Computer  Science  Tech. 
Rept.  TR-640,  Univ.  of  Maryland,  College  Park,  April  1978. 

Presentations 


Kanal,  Laveen  N.,  "Research  on  Hierarchical  Classifiers  and 
Interactive  Pattern  Analysis"  talks  presented  at:  Dept,  of 
Biomedical  Engineering,  Univ.  of  Missouri,  Columbia,  October 
17,  1977,  Dept,  of  Computer  Science,  Univ.  of  Nebraska,  Lincoln, 
October  20,  1977,  Dept,  of  Electrical  Engineering,  Univ.  of 
Southhampton,  Southhampton,  England,  November  30,  1977. 


Kanal,  Laveen  N.,  "Some  Current  Concepts  and  Problems  in  Pattern 
Classification  and  Feature  Extraction:,  invited  talk  on  December 
9,  1977  at  41st  meeting  of  the  International  Statistical  Institute, 
New  Delhi,  India. 

Kanal,  Laveen  N.,  talks  on  research  at  Indian  Statistical 
Institute,  New  Delhi,  during  December  1977. 

Kanal,  Laveen  N.,  "Current  Pattern  Recognition  Research  relevant 
to  Remote  Sensing",  talk  to  Remote  Sensing  Division,  Indian 
Space  Research  Organization  (ISRO),  AHmedabad,  India,  December 
26,  1977. 

Kanal, Laveen  N.  "Fundamentals  of  Pattern  Classification",  talk 
to  beginning  graduate  class,  Department  of  Math  and  Statistics, 
Gujrat  University,  Ahmbabad,  India,  Deceber  27,  1977. 
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PAPERS,  REPORTS,  AND  PRESENTATIONS 


KanaT,  Laveen  N.  , "Machine  Recognition  of  Patterns:  Potential 
for  Societal  Applications",  Invited  Fifth  Anniversary  Distinguished 
lecture,  ISRO,  Ahmedabad,  India  December  27,  1977. 


Kanal,  Laveen  N.,  "Descriptive  and  Generative  Models  for  0-1 
Processes  and  their  Application  to  Communication  Channels  with 
Memory",  talk  to  Communications  Division,  ISRO,  December  28,  1977. 

Kanal,  Laveen  N.,  Lectures  on  research  on  Interactive  Systems 
for  Pattern  Analysis,  Problem  Solving  and  Pattern  Recognition 
and  on  Severe  Storm  Pattern  Recogni ti on, at  Indian  Institute  of 
Science,  Bangalore,  Jan.  10-12,  1978. 

Kanal,  Laveen  N.,  Talk  to  Indian  Institute  of  Technology, 

Madras,  January  16,  1978,  on  pattern  recognition  research 
being  done  in  our  laboratory. 

Kanal,  Laveen  N.,  Talk  on  pattern  recognition  research  to  Computer 
Science  Department,  Jawaharlal  Nehru  University,  New  Delhi, 

January  20,  1978. 

Kanal  Laveen  N.,  "Problem  Solving  Search  Models  for  Pattern 
Recognition",  paper  presented  to  the  IEEE  Computer  Society 
Conference  on  Pattern  Recognition  and  Machine  Intelligence", 
Princeton,  April  13,  1978. 

Kanal,  Laveen  N.  , Lectures  on  Problem  Solving  and  Pattern 
Recognition,  Vrije  Uni versi tei t,  Amsterdam,  May  10,  17,  24, 

1978;  continuation  lecture  at  Delft  Institute  of  Technology, 

June  1,  1978.  These  lectures  were  attended  by  30-35  students 
and  faculty  representing  various  universities  and  institutes 
in  Holland. 

Kanal,  Laveen  N.,  Invited  talk  "Pitfalls  in  Applying  Pattern 
Recognition  Techniques  in  Practice",  special  symposium  American 
Chemical  Society,  Northeastern  Meeting  Boston,  Mass.,  June  27,  1978. 

Kanal,  Laveen  N.,  "On  the  Design  and  Evaluation  of  Medical 
Diagnosis  Models",  invited  presentation  at  special  symposium. 

Intern.  Conf.  on  Cybernetics  & Soc.,  Tokyo,  Nov.,  1978. 


Kanal,  Laveen  N.,  Admissible  Search  Strategies  for  Parametric 
and  Nonparametri c Hierarchical  Classifiers,"  presented  at  Fourth 
IJCPR,  Kyoto,  Nov.,  1978. 
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